For remote reconnaissance of the earth from satellites in the optical spectral range, optoelectronic line scanners are predominantly used. Either a line is scanned mechanically by a rotating mirror, or a single image line is simultaneously picked up by charge-coupled devices or CCD units. The cameras of the French Spot satellite and of the IRS satellite (Indian Remote Sensing Satellite) are examples. CCD line scanners for use with the shuttle or for a satellite include the (stereoscopic) modular optoelectronic multispectral scanner (MOMS or stereo MOMS), made by MBB, and the MEOSS scanners (for monocular electrooptical stereo scanner), made by the present assignee.
In all line scanners used thus far, the forward motion of the camera carrier, that is, the satellite or aircraft, produces the image from individual lines. The resultant continuous image strips are therefore not geometrically rigid. They are distorted by the dynamics of the carrier, and positional fluctuations and deviations from the ideal, straight-line forward motion are sources of error. For this reason, the geometric quality of the images obtained by means of line scanners is by no means satisfactory.
The image distortions can be detected only by a comparison with error-free images and maps, or with real positions of prominent landmarks on the surface of the earth. This requires a large number of comparison points, and their density must be adapted to the amplitudes and frequencies of the image disturbances.
If cameras that photograph two-dimensionally are used, the pictures taken with such cameras are two-dimensionally rigid, because the entire image area is photographed simultaneously. In that case, an overlapping pair of photographs can be looked at stereoscopically and evaluated accordingly. In this way, not only is a dimensionally stable copy (a so-called model) of the photographed surface obtained, but at the same time the location and angular orientation of the two still cameras relative to the surface that is copied are ascertained as well.
For the last-mentioned purpose, the image positions of four points, not located on a single line, in the overlapping zone of the pair of pictures taken by the two cameras are sufficient. This is known as the "self-orientation capacity" of such still cameras. A prerequisite is that the invariable copying properties of the camera be accurately known.
The self-orientation capacity of still cameras is utilized in aerial mapping to produce large coherent composite images from overlapping single photographs. By using four comparison points per overlapping zone at a time, all the images can be oriented relative to one another, and all the cameras can be oriented relative to the actual earth surface, with a single uniform scale. This is known as "model resolution" in "photogrammetric block balancing".
For a "model resolution" of this kind, it is sufficient to compare the positions of identical image points in the image space (that is, on the individual photographs). The true, actual image coordinates of these points are not needed for forming the model. However, the ground coordinates of only four points (for instance in the corners of a block) are sufficient to ascertain the location and scale of the ideal surface from the model (that is, from the block).
For orienting line scanners with respect to the earth's surface, still cameras are sometimes used parallel with the line scanners in aircraft. Problems then arise, however, because both digital and analog (photographic) data need to be processed further, and not only must the image data be temporally associated with one another, but the picture repetition frequency for the photographs taken must be high, so that the high-frequency carrier dynamics resulting, for instance, from jarring, natural oscillation, and the like can be detected.
In video cameras equipped with CCD area sensors and so far generally intended for use by amateurs, the area sensors typically have 250.times.500 pixels (columns and lines). Remote reconnaissance and mapping of the earth, however, requires cameras having approximately 10,000.times.10,000 pixels, yet the production of malfunction-free, homogeneous area detectors of this order of magnitude in the foreseeable future is impossible. Attempts have therefore been made to produce image areas having the required number of pixels by means of a mosaic arrangement of smaller CCD area detectors.
For technical reasons, especially the number of supply lines and the need to carry signals, a gap-free mosaic arrangement of this kind is impossible. To cover the gaps, at least one further parallel-oriented camera with a suitably offset mosaic must therefore be used.
The primary problem with large-area optoelectronic cameras, however, is in data transmission. To keep the transmission rate constant, the entire image content must be stored in memory until the next photograph is taken. Thus far, the only possible memories for this purpose were the CCD units themselves, but these units have relatively high electronic noise and dark currents, so that the picture quality suffers from the long memory storage times (of up to one minute). This kind of arrangement of optoelectronic solid-state sensor areas in photogrammetric copying systems is described in German patent 34 28 325, for example.
The overlapping pictures taken with area cameras enable stereoscopic viewing and surveying of the terrain, as well as the above-described model formation by block balancing for large composite images, and for this reason are preferred in topographic mapping.
Multispectral or panchromatic line scanners, however, are used to a substantially greater extent and therefore substantially more widely; their true- or false-color images allow thematic classification of the surface (for example by types of vegetation or soil). In that case, though, the view direction must be as uniform and as parallel to the sun as possible; otherwise the color relationships are shade-dependent or in other words have a bluish cast.
The fact that in area cameras the view direction also varies with the direction of flight has also led to a clear preference for line scanners for multispectral mapping. For this reason, CCD area sensors for multispectral photographs are also preferentially used in prism or grating spectrometers. Here, only one line on the earth's surface is photographed at a time, that is, in one exposure. This single line, however, is spectrally decomposed by a prism or a grating, so that the spectral information is distributed onto the area CCD. This means that each line of the area detector sees a different "color" of the same line on the ground.
In terms of the development of optoelectronic line scanners with stereoscopic capacity, a number of proposals for making this type of equipment usable for topographic mapping as well have already been published. Examples include the cameras for a project equipped with MBB stereo MOMS and the MEOSS Project, thus far the only one carried out, of the present assignee.
For stereoscopic pictures, at least two planes inclined relative to one another must be used. This can be done by tilting complete line scanners, for instance with stereo MOMS, or by using parallel CCD line detectors in the image plane of optical equipment, as in the case of the MOESS Project. To enable mutual orientation, or in other words a model self-orientation like that in photogrammetric block formation from area photographs, however, at least three scanning planes must be used; this is schematically illustrated in FIG. 1.
A single image in this case comprises three lines, exposed at the same time, which are are spaced apart by a socalled "basic length" B. An instantaneous image of this kind is called a "line triplet". Under ideal flying conditions, that is, a constant height and speed, with the camera oriented parallel to the ground, and so forth, the individual pictures, i.e., the line triplets, each spaced apart by the basic length B, then overlap. In an ideal case such as this, four (or usually, more than four) common points are then located on each of two lines of both line triplets, and these points make it possible to ascertain the mutual "ideal orientation" of the triplets to one another.
Generally, however, four common points for two line triplets cannot be found, because the pixels of one line triplet, spaced apart by the basic length B, are distributed over a certain zone known as a coherent segment of a plurality of line triplets. As a result, it is no longer possible to orient the line triplets relative to one another in a mathematically univocal way.
Adjacent line triplets can, however, be combined into so-called "segment triplets". In that case, then four or more common pixels can always be found for a mutual orientation of the segment triplets spaced apart by the basic length B.
Since segment triplets are formed from line triplets that are exposed in chronological succession, however, they themselves are already distorted from fluctuations in the path and location of the camera carrier, i.e., the aircraft or satellite. Then they are no longer located in the same common image plane, such as that shown schematically in FIG. 2, and despite the presence of four common pixels, they can no longer be mathematically univocally oriented to one another.
To arrive at acceptable approximations that are acceptable for practical use, various interpolation models have been used. In its MEOSS Project, for instance, the present assignee developed and used satellite-specific theoretical models of the path and location dynamics as interpolation models.
Nevertheless, the methods and apparatus used thus far have various disadvantages. In the optoelectronic area cameras, it is disadvantageous that different view directions develop in the flight direction; that many individual area detectors must be adjusted for a large mosaic; that two or more parallel-oriented optical systems, i.e., cameras, must be used; and that the contents of a complete image must be stored in buffer memory for a relatively long period of time.
In orientation with stereoscopic three-line scanners, it is disadvantageous that interpolation models are required; that software developed for photogrammetric purposes must be expanded for block balancing; that the line scanners are sensitive to high frequency carrier dynamics; that long image strips are required, or else the area being photographed must be flown over in a zigzag pattern; and finally, that if the individual lines are canted relative to one another, the height resolution in stereoscopic evaluation is not constant.